<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.14"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>Logagent: Logagent - Multi-channel Log Collector</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectalign" style="padding-left: 0.5em;">
   <div id="projectname">Logagent
   </div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.14 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
var searchBox = new SearchBox("searchBox", "search",false,'Search');
/* @license-end */
</script>
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
$(function() {
  initMenu('',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
/* @license-end */</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">Logagent - Multi-channel Log Collector </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><div class="image">
<object type="image/svg+xml" data="https://img.shields.io/travis/rust-lang/rust.svg" alt="build"></object>
</div>
<div class="image">
<object type="image/svg+xml" data="https://img.shields.io/github/license/mashape/apistatus.svg" alt="license"></object>
</div>
<p>Logagent is designed to collector logs and send to specified places.</p>
<h2>Framework introduction</h2>
<p>Logagent work with extensible plugins. similar to the <a href="https://gstreamer.freedesktop.org/">Gstreamer</a>, each log stream have a independent</p>
<p>working channel which called <code>pipeline</code>, <code>pipeline</code> is make up by mulit <code>element</code>, <code>element</code> is a copy</p>
<p>of the <code>plugin</code> which maintained by the plugin store.</p>
<h4>pipeline</h4>
<p>pipeline is the log channel of the Logagent framework. Each pipeline have a independent thread to run.</p>
<p><code>pipeline</code> maintained a <code>element</code> list, when the element list is prepared, logagent create a thread to run</p>
<p>pipeline, and call element's work function one by one. Each working period, all elements share a log buffer</p>
<p>list, so the first element maybe a collector to add log into the log buffer; the second one maybe a log filter</p>
<p>to remove some logs from the log buffer; the last one maybe a log sender which send logs to the specified</p>
<p>place.</p>
<h4>element</h4>
<p>element is the smallest component of the pipeline, each element saved the location of it's plugin. Each time</p>
<p>to call the element's function, find the current plugin from the plugin store and call the plugin's thread</p>
<p>safety work function.</p>
<h4>plugin</h4>
<p>plugin is a dynamic library which provides functions called by logagent. Logagent maintained a plugin store,</p>
<p>after config loading, plugin initialize global environment and save global config to plugin store. Element call</p>
<p>plugin <code>init</code> function and save element's independent config to element's own places.</p>
<h2>Project dependence</h2>
<p>Logagent:</p>
<p>​ <a href="https://curl.haxx.se/libcurl/">libcurl</a>: the multiprotocol file transfer library</p>
<p>​ <a href="https://github.com/json-c/json-c.git">json-c</a> : A JSON implementation in C</p>
<p>Plugins:</p>
<p>​ <a href="http://www.man7.org/linux/man-pages/man7/inotify.7.html">inotify</a>: monitoring filesystem events</p>
<p>​ <a href="https://github.com/aliyun/aliyun-log-c-sdk.git">aliyun-log-c-sdk</a>: Aliyun LOG Producer for C/C++</p>
<h2>Build instruction</h2>
<p>install all dependences:</p>
<blockquote class="doxtable">
<p>sudo apt install libjson-c-dev libcurl3 libapr1-dev libaprutil1-dev </p>
</blockquote>
<p>clone the repo:</p>
<blockquote class="doxtable">
<p>git clone <a href="https://github.com/geekerlw/logagent.git">https://github.com/geekerlw/logagent.git</a> </p>
</blockquote>
<p>init and pull submodules:</p>
<blockquote class="doxtable">
<p>git submodule init</p>
<p>git submodule update </p>
</blockquote>
<p>build like all project maintained by autotools:</p>
<blockquote class="doxtable">
<p>./autogen.sh</p>
<p>./configure # &ndash;prefix=/usr</p>
<p>make &amp;&amp; make install </p>
</blockquote>
<h2>Configuration and usage</h2>
<h3>how to write configure json file</h3>
<p>Logagent use json to config <code>pipeline</code> <code>element</code>and <code>plugins</code>.</p>
<p>example:</p>
<div class="fragment"><div class="line">{</div><div class="line">    &quot;pipeline_nums&quot;: 2,</div><div class="line">    &quot;pipeline@0&quot;: {</div><div class="line">        &quot;element_nums&quot;: 2,</div><div class="line">        &quot;element@0&quot;: {</div><div class="line">            &quot;plugin_name&quot;: &quot;filesrc&quot;,</div><div class="line">            &quot;filesrc_config&quot;: &quot;your plugin config&quot;</div><div class="line">        },</div><div class="line">        &quot;element@1&quot;: {</div><div class="line">            &quot;plugin_name&quot;: &quot;aliyun&quot;,</div><div class="line">            &quot;aliyun_config&quot;: &quot;your plugin config&quot;</div><div class="line">        }</div><div class="line">    },</div><div class="line">    &quot;pipeline@1&quot;: {</div><div class="line">        &quot;element_nums&quot;: 2,</div><div class="line">        &quot;element@0&quot;: {</div><div class="line">            &quot;plugin_name&quot;: &quot;filesrc&quot;,</div><div class="line">            &quot;filesrc_config&quot;: &quot;your plugin config&quot;</div><div class="line">        },</div><div class="line">        &quot;element@1&quot;: {</div><div class="line">            &quot;plugin_name&quot;: &quot;aliyun&quot;,</div><div class="line">             &quot;aliyun_config&quot;: &quot;your plugin config&quot;</div><div class="line">        }</div><div class="line">    },</div><div class="line"></div><div class="line">    &quot;plugin_nums&quot;: 2,</div><div class="line">    &quot;plugin@1&quot;: {</div><div class="line">          &quot;plugin_name&quot;: &quot;filesrc&quot;,</div><div class="line">          &quot;plugin_path&quot;: &quot;/usr/lib/logagent&quot;</div><div class="line">    },</div><div class="line">    &quot;plugin@2&quot;: {</div><div class="line">          &quot;plugin_name&quot;: &quot;aliyun&quot;,</div><div class="line">          &quot;plugin_path&quot;: &quot;/usr/lib/logagent&quot;</div><div class="line">    }</div><div class="line">}</div></div><!-- fragment --><p>pipeline:</p>
<table class="markdownTable">
<tr class="markdownTableHead">
<th class="markdownTableHeadCenter">key  </th><th class="markdownTableHeadCenter">description  </th><th class="markdownTableHeadCenter">value   </th></tr>
<tr class="markdownTableBody" class="markdownTableRowOdd">
<td class="markdownTableBodyCenter">pipeline_nums  </td><td class="markdownTableBodyCenter">pipeline count, each pipeline correspond a thread  </td><td class="markdownTableBodyCenter">int   </td></tr>
<tr class="markdownTableBody" class="markdownTableRowEven">
<td class="markdownTableBodyCenter"><code>pipeline@x</code>  </td><td class="markdownTableBodyCenter">pipeline objects, x start from 0 to pipeline -1  </td><td class="markdownTableBodyCenter">json objects   </td></tr>
</table>
<p>element:</p>
<table class="markdownTable">
<tr class="markdownTableHead">
<th class="markdownTableHeadCenter">key  </th><th class="markdownTableHeadCenter">description  </th><th class="markdownTableHeadCenter">value   </th></tr>
<tr class="markdownTableBody" class="markdownTableRowOdd">
<td class="markdownTableBodyCenter">element_nums  </td><td class="markdownTableBodyCenter">element count, each pipeline contained an elements object  </td><td class="markdownTableBodyCenter">int   </td></tr>
<tr class="markdownTableBody" class="markdownTableRowEven">
<td class="markdownTableBodyCenter"><code>element@x</code>  </td><td class="markdownTableBodyCenter">element objects, x start from 0 to element_nums -1  </td><td class="markdownTableBodyCenter">json object   </td></tr>
<tr class="markdownTableBody" class="markdownTableRowOdd">
<td class="markdownTableBodyCenter">plugin_name  </td><td class="markdownTableBodyCenter">under element, given a plugin name match with the plugin objects  </td><td class="markdownTableBodyCenter">string   </td></tr>
<tr class="markdownTableBody" class="markdownTableRowEven">
<td class="markdownTableBodyCenter">...  </td><td class="markdownTableBodyCenter">element's personal config  </td><td class="markdownTableBodyCenter">customiz   </td></tr>
</table>
<p>plugin:</p>
<table class="markdownTable">
<tr class="markdownTableHead">
<th class="markdownTableHeadCenter">key  </th><th class="markdownTableHeadCenter">description  </th><th class="markdownTableHeadCenter">value   </th></tr>
<tr class="markdownTableBody" class="markdownTableRowOdd">
<td class="markdownTableBodyCenter">plugin_nums  </td><td class="markdownTableBodyCenter">plugin count, each plugin contained a plugin  </td><td class="markdownTableBodyCenter">int   </td></tr>
<tr class="markdownTableBody" class="markdownTableRowEven">
<td class="markdownTableBodyCenter"><code>plugin@x</code>  </td><td class="markdownTableBodyCenter">plugin objects, x start from 0 to plugin_nums -1  </td><td class="markdownTableBodyCenter">json object   </td></tr>
<tr class="markdownTableBody" class="markdownTableRowOdd">
<td class="markdownTableBodyCenter">plugin_name  </td><td class="markdownTableBodyCenter">library load match with liblogagent-plugin-<code>plugin_name</code>.so  </td><td class="markdownTableBodyCenter">string   </td></tr>
<tr class="markdownTableBody" class="markdownTableRowEven">
<td class="markdownTableBodyCenter">plugin_path  </td><td class="markdownTableBodyCenter">plugin library path  </td><td class="markdownTableBodyCenter">string   </td></tr>
<tr class="markdownTableBody" class="markdownTableRowOdd">
<td class="markdownTableBodyCenter">...  </td><td class="markdownTableBodyCenter">plugin's global config  </td><td class="markdownTableBodyCenter">customiz   </td></tr>
</table>
<h3>how to use</h3>
<p>try logagent &ndash;help for help information</p>
<blockquote class="doxtable">
<p>logagent &ndash;help </p>
</blockquote>
<p>load json config from local disk:</p>
<blockquote class="doxtable">
<p>logagent -f /etc/logagent/logagent.conf </p>
</blockquote>
<p>load json config from network:</p>
<blockquote class="doxtable">
<p>logagent -n <code><a href="https://your_json_config_url">https://your_json_config_url</a></code> </p>
</blockquote>
<h2>Documentation and development</h2>
<h3>plugins description</h3>
<h5>filesrc</h5>
<p>use inotify to monitor directory, collect matching log.</p>
<p>how to config:</p>
<p><code>file_path</code>: the log directory you want to monitor</p>
<h5>aliyun</h5>
<p>use <a href="https://github.com/aliyun/aliyun-log-c-sdk.git">aliyun-log-c-sdk</a> to send log to aliyun log server</p>
<p>how to config:</p>
<p><code>logstore_config</code>: global config json file fullpath, see https://github.com/aliyun/aliyun-log-c-sdk/blob/master/README.md "readme" for more help.</p>
<p><code>logstore_name</code>: set under element's config to match logstore_config.</p>
<h3>how to write a plugin</h3>
<p>Each plugin need to provide those functions defined in <a href="./plugins/logagent-plugin-api.h">logagent plugin api</a></p>
<div class="fragment"><div class="line">extern int logagent_plugin_env_init(void **context);</div><div class="line">extern int logagent_plugin_env_destroy(void **context);</div><div class="line">extern int logagent_plugin_work(void *gconfig, void *pconfig,</div><div class="line">                struct list_head *log_list);</div><div class="line">extern int logagent_plugin_init(void *gconfig, void **context);</div><div class="line">extern int logagent_plugin_exit(void *gconfig, void **context);</div></div><!-- fragment --><p>see the <a href="./docs/html/index.html">full documentation</a> for more help.</p>
<h2>Licence</h2>
<p><a href="https://github.com/geekerlw/logagent/blob/master/LICENSE">MIT Licence</a></p>
<p>The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. </p>
</div></div><!-- contents -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/>
</a> 1.8.14
</small></address>
</body>
</html>
